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ABSTRACT 

During the 2006-2007 academic year, five faculty members from the College of Engineering 
at Boise State University initiated a curriculum augmentation project using new instructional 
technologies with the intention to help improve undergraduate engineering students’ cognitive 
and affective preparedness for their classroom learning. The instructional technologies used in 
the project were a pre-instructional strategy and a self-paced e-learning method. The main ques¬ 
tion addressed in this project was: Will a pre-instructional e-learning strategy help engineering 
students cognitively and affectively prepare for their classroom learning? This paper is a report of 
the project, describing the analysis, design, and development of a multimedia e-learning module 
for an engineering curriculum, the implementation of the e-learning module as a pre-instructional 
strategy in two engineering courses, and the evaluation of the effectiveness of using the pre-in¬ 
structional e-learning strategy on engineering students’ cognitive and affective preparedness for 
classroom learning. It also provides a list of lessons learned from the project. 

Keywords: e-learning, pre-instructional strategy, reusable learning objects. 
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I. INTRODUCTION 


Despite the steady growth of overall enrollment in Science, Technology, Engineering, and 
Mathematics (STEM) education in the United States during the last three decades, enrollment in 
engineering programs in the nation still remains low [1, 2], The lack of participation in engineering 
programs is attributed to various interrelated factors such as a lack of academic readiness in math, 
a failing experience during a critical stage of learning, misperceptions about an engineering career, 
a misunderstanding of the relationship between early engineering courses and a future engineering 
career, poor academic advisement, and a lack of social support [3-7], These interrelated factors dur¬ 
ing students’ early stage of learning in engineering call for special attention, but there is no single 
solution for this multifaceted problem. Resolution requires implementation of effective interventions 
in many dimensions of the educational system, such as institutional support, faculty development, 
and quality curriculum design. 

From the curriculum design standpoint, effective instructional technologies should be integrated 
into the engineering curricula to help students become successful learners while they are preparing 
to pursue careers in engineering. With that in mind, a curriculum augmentation project was initiated 
in the College of Engineering at Boise State University during the 2006-2007 academic year. The 
project focused on improving cognitive and affective preparedness of undergraduate engineering 
students for classroom learning by implementing new instructional technologies including a pre- 
instructional strategy designed with reusable learning objects and a self-paced e-learning method. 
The following section provides the theoretical framework of this project. 


II. PREPARING LEARNERS WITH INSTRUCTIONAL TECHNOLOGIES 


A. The Importance of Cognitive and Affective Preparedness 

Instruction is a set of systematically arranged events, designed to facilitate learning processes 
and to produce desired learning outcomes [8]. Ideally, when creating an instructional lesson or a 
curriculum, the instructional designer analyzes the types of instructional content based on targeted 
learning outcomes before designing the events of instruction [9], Bloom and his colleagues de¬ 
fined three types (domains) of learning outcomes in their taxonomy of educational objectives—the 
cognitive domain (thinking), the affective domain (feeling), and the psychomotor domain (acting) 
[10, 11]. Since its development in the 1950s, Bloom’s taxonomy has been adopted and elaborated 
by other instructional theorists and widely used in curriculum design [e.g., 12, 13], as it provides 
instructional designers with a comprehensive understanding about the types of learning outcomes 
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produced by students, and it helps them develop and deliver instruction in a systematic manner. 
The three domains are intertwined such that positive outcomes in one domain often require support 
from others (see Figure 1). The cognitive domain deals with the development of intellectual abilities 
and skills, and the affective domain deals with the development of appreciation and motivation of 
learning [10,11]; therefore, these two domains are particularly important in most subjects in higher 
education including engineering. 

The cognitive and affective domains of learning outcomes are divided into several hierarchi¬ 
cal levels, which are used to develop an instructional sequence from simple to more difficult and 
complex levels [13]. For example, the cognitive domain contains six hierarchical levels: knowledge, 
comprehension, application, analysis, synthesis, and evaluation [10], and the affective domain con¬ 
tains five hierarchical levels: receiving, responding, valuing, organization, and characterization by 
one’s value system [11], When students learn a new subject, they begin from lower levels of cognitive 
learning, knowledge and comprehension (i.e., knowing what it is), supported by receiving, respond¬ 
ing, and valuing levels of affective learning (i.e., wanting to learn) before moving onto the higher, 
more complex levels of learning. In this early stage of learning, students also start developing their 
self-efficacy, which not only is a self-concept of one’s cognitive abilities but also involves affective 
states influenced by the person’s emotional experiences [14], Therefore, the instructional events of 
an engineering curriculum would be most effective when they are designed to help students build 
cognitive and affective foundations to prepare for the subsequent learning processes. In other words, 
the learning environment should result in the student thinking, “I know I can do it, I’m confident that 
I will be successful, and I’ll continue to learn it.” 

B. The Use of a Pre-lnstructional Strategy 

One such instructional method that is designed to facilitate the development of cognitive and 
affective preparedness for classroom learning is a pre-instructional strategy. Instructional events 



Figure 7; Three domains of learning outcomes (an interactive version of Figure 7 is 
available at the following URL - http://advances.asee.org/vol02/issue01/media/13/Figure1/ 
engage. htmlj . 
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are purposely arranged to maximize the learning outcomes. One may assume that purposely and 
systematically arranged instructional events occur only inside the classroom. However, the effective¬ 
ness of instruction can be improved by starting with a structured pre-instructional phase outside the 
classroom. An overview type of pre-instruction alerts learners of new content and prepares them 
to fully process new information during actual classroom instruction [15]. When teaching novice 
learners in the classroom, the pre-instructional overview is best designed for the knowledge and 
comprehension levels of the cognitive domain. Knowledge of facts, definitions of new concepts, and 
explanations of principles are necessary to improve their classroom learning. Successful completion 
of the pre-instructional overview can also improve the students’ affective states of preparedness 
for classroom learning; that is, the students will enter the classroom feeling prepared, confident, 
and motivated to learn. 

Compared to unstructured pre-instructional strategies such as giving reading assignments or 
post-instructional strategies such as a drill-and-practice type of homework, the benefits of using 
structured pre-instruction in engineering courses are promising. At the novice learner level, the 
pre-instructional strategy would be most effective if focused on an overview of the material at the 
lower levels of Bloom’s taxonomy. Only a small number of research studies have been reported on 
the topic of the effects of structured pre-instructional strategies used in conjunction with, and prior 
to, traditional classroom teaching on learning improvement [16, 17], One study [17] revealed that 
when students started utilizing a more intensive pre-laboratory activity in chemistry courses (from 
20 minutes of unstructured preparation to one hour and 50 minutes of structured pre-lab instruc¬ 
tion), a majority of them liked the new instructional structure, although some of them felt negatively 
about the increased workload before the lab. Forty-one percent of the students indicated that this 
new system might be suitable for difficult subjects. 

C. The Use of Self-Paced E-Learning Designed with Reusable Learning Objects 

Instructional methods shape the instructional content, and instructional media provide the delivery 
support. A pre-instructional strategy is an instructional method, and it can be effectively delivered 
via computer technology coupled with the Internet and the World Wide Web, which is known as 
e-learning. Presently, e-learning is deeply integrated into school curricula to motivate students and 
facilitate learning [18], A growing body of literature exists in the community of engineering educa¬ 
tion that indicates the benefits of incorporating e-learning strategies in engineering education [e.g., 
19-26]. In fact, leading academic organizations such as the Sloan Consortium advocate that several 
of the ABET engineering competencies can be augmented by online learning [19]. In contrast to 
one-to-many classroom learning, e-learning allows learners to adjust the pace, sequence, and type 
of media to better fit their learning behavior and needs. A number of studies have shown positive 
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results when using self-paced computer-supported learning environments [e.g., 27, 28], A tutorial, 
overview-type of pre-instructional e-learning allows self-paced learning during which students can 
increase confidence and better prepare for classroom learning. 

One aspect to be considered when implementing a pre-instructional strategy through e-learning 
is to design the content with reusable multimedia learning objects to increase efficiency. A learning 
object (LO) is “any entity, digital or non-digital, which can be used, reused or referenced during 
technology supported learning” [29], Granularity is the core principle behind the design and use 
of LOs. Although LOs may come in different sizes, each LO is usually developed based on a single 
learning objective, and several LOs can be combined to form a larger scale instructional unit such as 
a unit or a complete course [30-32], Benefits of designing e-learning content with the concept of 
LOs in mind include reusability, consistency, and cost effectiveness. LOs can be developed once and 
assembled differently to be used in multiple courses, resulting in reusable learning objects (RLOs) 
which improve consistency and cost-effectiveness in curriculum design. 

D. Putting it All Together 

Acknowledging the importance of developing engineering students’ cognitive and affective prepared¬ 
ness for their classroom learning during their early learning processes and the benefits of using a pre- 
instructional e-learning strategy, the authors initiated a curriculum augmentation project by developing 
the described instructional technologies. The main goal of the project was to help engineering students 
improve their cognitive and affective preparedness for classroom learning. The project was partially 
supported by the National Science Foundation grant, New Bachelor’s Degree in Materials Science and 
Engineering, awarded to Boise State University in 2006. Figure 2 illustrates the triadic relationship among 
the three elements considered in this project with the goal of producing successful learning. 

The project was conducted through a systematic instructional development process called ADDIE, 
which is an acronym that refers to five major steps—analysis, design, development, implementation, 


Curriculum design 
(pre-instructional e- 
learning) 



Learning environment 
(web-based, self-paced) 


Figure 2: The triadic relationship among learners, curriculum design, and the learning 
environment (an interactive version of Figure 2 is available at the following URL - http:// 
advances.asee.org/vol02/issue01/media/13/Fiqure2/enqaqe.htmlJ . 
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and evaluation [33], The following is a report of the project, describing the systematic instructional 
design principles that were followed, the actual multimedia e-learning module that was developed, 
and the evaluation results that were obtained. 


III. DESIGN AND DEVELOPMENT OF E-LEARNING 


A. Analyzing Learners 

One of the early steps in systematic instructional development is to analyze learner character¬ 
istics. Of particular interest in this project was how different age groups of engineering students 
would use a multimedia e-learning module as a pre-instructional tool. Given the increase in non- 
traditional students participating in undergraduate programs in recent years [34], research that 
addresses differences in studying engineering as a function of age of the student is important 
for the enhancement of engineering education. This issue is particularly important for the four 
undergraduate engineering programs at Boise State University because of the high population 
of non-traditional students (more than 30% are older than 25). Boise State has approximately 
750 undergraduate students enrolled in Civil Engineering, Electrical and Computer Engineering, 
Mechanical and Biomedical Engineering, and Materials Science and Engineering, and most of the 
engineering classrooms consist of a mixture of younger and older students. However, very little 
research has been conducted on the effect that age has on learning, especially e-learning, in STEM 
subjects at the undergraduate level. A small number of studies have shown that no significant 
differences exist in academic achievement or completion rates between traditional and non-tradi- 
tional undergraduate students [e.g., 35-37], However, in terms of computer use, older users report 
lower levels of confidence in using computers, compared to younger users [38, 39], This aspect of 
age difference coincided with anecdotal evidence observed at Boise State that age is potentially 
an important factor that influences engineering students’ overall behaviors and attitudes toward 
their learning. That is, non-traditional older students tend to take learning more seriously, but they 
typically have more demands on their time (e.g., families and work); hence, they try to be more 
efficient about how they prepare for class. However, younger and older students also have differ¬ 
ent attitudes toward using a computer as a learning tool. Therefore, the authors paid attention to 
potential age difference in the evaluation of the project, to investigate how younger students and 
older students in engineering classrooms would perform in an e-learning environment and what 
impact a pre-instructional e-learning strategy would have on improving their cognitive and affec¬ 
tive preparedness for classroom learning. 


6 


SPRING 2010 





ADVANCES IN ENGINEERING EDUCATION 

Improving Engineering Students’ Cognitive and Affective 
Preparedness with a Pre-lnstructional E-Learning Strategy 



B. Analyzing Reusable Learning Objects 

With reusability in mind, it was decided to select the content of e-learning modules (LOs) based 
on the Introduction to Materials Science and Engineering class (ENGR 245) taken by all engineer¬ 
ing majors. The LOs would be reusable in higher-level courses that build upon the prerequisite 
knowledge gained in the introductory course. For example, a LO on the topic of atomic bonding 
developed for the introductory course can be reused in a ‘modern physics’ class and a 'bonding 
and crystallography’ class as refresher material. A LO on a topic of vectors can be used not only 
in the introductory class but also in a ‘statics’ class, an ‘engineering mechanics of materials’ class, 
and a ‘bonding and crystallography’ class. With no or minimal supervision by an instructor, stu¬ 
dents in advanced classes can have an opportunity to review the prerequisite materials to prepare 
themselves to continue with higher-level learning. This reusability can also increase consistency in 
instructional contents and methods across different sections of the course and instructors. Figure 3 
presents a simplified illustration of reusability analysis of seven specific LOs, and Table 1 shows a 
more detailed explanation. 

C. Developing a Multimedia E-Learning Module 

The e-learning development team consisted of a project coordinator, subject matter experts 
(engineering faculty), and e-learning developers (graduate assistants). Among the reusable learning 
objects that were analyzed (as shown in Table 1), the first e-learning module on the topic of ‘Mechanical 



Figure 3: Developing RLOs of the Introduction to Materials Science & Engineering course. 
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LO Topics in ENGR 245 

Other STEM courses in which the LO will be reused 

The Arrhenius Relationship 

- EE 323 Semiconductor Devices 

- MSE 305 Bonding, Crystallography, and Crystal Defects 

- MSE 308 Thermodynamics of Materials 

-MSE 310 Electrical Properties of Materials 

- MSE 408 Phase Transformations and Kinetics 

Band Theory and Electrical 

- EE 323 Semiconductor Devices 

Properties 

- MSE 310 Electrical Properties of Materials 

- PHYS 309 Modern Physics 

- PHYS 415 Solid State Physics 

Atomic Bonding 

- MSE 305 Bonding, Crystallography, and Crystal Defects 

- PHYS 309 Modem Physics 

Crystallography 

- EE 323 Semiconductor Devices 

- MSE 305 Bonding, Crystallography, and Crystal Defects 

- MSE 310 Electrical Properties of Materials 

- MSE 312 Mechanical Behavior of Materials 

- MSE 505 Bonding and Structure of Materials 

Dislocations 

- ENGR 350 Engineering Mechanics of Materials 

- MSE 305 Bonding, Crystallography, and Crystal Defects 

- MSE 312 Mechanical Behavior of Materials 

- MSE 505 Bonding and Structure of Materials 

Mechanical Properties of 

- CE 352 Structures 

Metals 

- ENGR 306 Mechanics of Materials 

- ENGR 350 Engineering Mechanics of Materials 

- MSE 312 Mechanical Behavior of Materials 

- MSE 512 Mechanical Properties of Materials (graduate- 

level course) 

Vectors 

-ENGR 210 Statics 

- ENGR 220 Dynamics 

- ENGR 306 Mechanics of Materials 

- ENGR 350 Engineering Mechanics of Materials 

- MSE 312 Mechanical Behavior of Materials 

- MSE 305 Bonding, Crystallography, and Crystal Defects 

All of the LOs above 

- MSE 600 Comprehensive Exam 

- Fundamentals for Engineering Exam for Licensure 


Table 1: A list of STEM courses in which the LOs can be reused. 
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Properties of Metals' was developed in 2006. This e-learning module designed with multimedia (text, 
images, animation, video, and audio) was developed based on Gagne’s nine events of instruction [9] 
and Cisco’s guidelines for designing RLOs [32], using e-learning development software, Articulate®. 
As shown in Table 2, the e-learning module consisted of a module overview, instructional topics with 
practice, and a module test of 30 questions including multiple-choice, true-false, drag-and-drop 
matching, drag-and-drop sequencing, and short-answer questions. The instruction with audio nar¬ 
ration was presented at a pre-set pace, but students could also control the sequence by using the 
menu on the left side of the screen. At the end of the module, a survey was presented to measure 
students’ feelings of preparedness, confidence, and attitude toward using e-learning (see Appendix 
A). It was estimated that students could complete the module in an hour. 


Gagne’s nine events 

of instruction 

Cisco’s RLO structure 

The Contents of the E- 

Learning Module 

1. Gain attention 

Introduction, 

Title 

2. Inform learners of 

relevance, objectives, 

Module overview 

objective 

prerequisites, scenario 


3. Stimulate recall of 

prior knowledge 

or outline 


4. Present new content 

A series of topics with 

Topic 1: Stress & Strain 

5. Provide learning 

practices: e.g., 

Loading types 

guidance 

Topic 1 + practice 

Engineering stress 

6. Elicit performance 

Topic 2 + practice 

Engineering strain 

7. Provide feedback 

Topic 3 + practice 

Sample calculations 


etc. 

(practice questions) 

Tensile tests 

Topic 2: Tensile 

Properties 

Hooke’s law 

Poisson’s ratio 

Yielding 

Yield strength 

Elastic region 

Tensile strength 

Necking 

Fracture strength 

Ductility 

Toughness 

Design safety factor 

8. Assess performance 

Assessment 

Module test 

9. Enhance retention 

and transfer 

Summary review 

Module survey 


Table 2: The design of the e-learning module. 
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This tutorial-type, pre-instructional e-learning module was designed mainly to facilitate knowl¬ 
edge and comprehension levels of learning, which can be used in different ways depending on the 
levels of learners. It can help novice learners prepare for the new information before their classroom 
learning, and it can help advanced learners to recall the prerequisite knowledge necessary for their 
more advanced learning. Figure 4 depicts a high-level concept model about the different uses of 
the e-learning module based on the levels of learners. 

Figure 5 and Figure 6 are screen shots of the e-learning module and the tensile test demonstra¬ 
tion video clip used in the module. The actual multimedia e-learning module and the video clip are 
also available from the web URLs. 


IV. EVALUATION I: PRE-INSTRUCTIONAL E-LEARNING (FALL 2006) 


A. Methods 

1) Purpose: The e-learning module was implemented in the ENGR 245 Introduction to Materials 
Science and Engineering course in the fall semester of 2006 as a pre-instructional strategy, and its 
effectiveness for preparing students to learn during the classroom lecture was investigated. The 
independent variable used in the evaluation was age, and the dependent variables were the degrees 
of students’ cognitive preparedness measured by the module test scores and affective preparedness 
measured by the module survey (see Appendix A) and a post-lecture survey (see Appendix B). 

2) Participants: Seventy-six students were enrolled in the course, and 50 of them voluntarily par¬ 
ticipated in the study. Forty-one students (82%) were male and nine (18%) were female. Participants’ 
ages ranged between 19 and 54. The average age was 25.78 and the median age was 24. Two age 









Evaluation 
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k 

Evaluation 
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Synthesis 



Synthesis 


Classroom 


Analysis 


Classroom 

Analysis 


learning 


Application 


learning 

Application 




Comprehension 

] Acquisition 

V- of new 


Comprehension 

> 

1 

1 

N 

Recall of 
prerequisite 
knowledge 
through e- 
leaming 


Knowledge 

| knowledge 
through 
e-leaming 


Knowledge 

1 

1 

1 

Novice learners 


Advanced learners 

Figure 4: Different uses of the e-learning module depending on the learner levels. 
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Figure 5: A screen shot of the e-learning module on ‘Mechanical Properties of Metals.’ 
(The actual e-learning module is available at http://advances.asee.org/vol02/issue01/ 
media/13/Module/launcher.htmL ) 



Figure 6: A screen shot of the tensile test demonstration video. (The actual video is 
available at http://advances.asee.orq/vol02/issue01/media/13/stressstrainvideo/demo.htmlj 
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groups were formed by using the median age as the cut point—a younger group (24 years old or 
younger) and an older group (older than 24 years of age). Table 3 shows the demographic breakdown 
by age and gender. 

3) Instruments and Procedure: The e-learning module was posted on a course web site, and 
participants were asked to complete it within three days prior to the classroom lecture that covered 
the topic. At the end of the module, a module test was administered to measure students' cogni¬ 
tive preparedness, and a module survey (Appendix A) was administered to measure their affective 
preparedness such as their feelings of preparedness for the lecture (Ql), confidence levels (Q2), and 
preference toward using e-learning (Q3 and Q4). After the classroom lecture, a post-lecture survey 
(Appendix B) was administered to ask students’ after-thoughts about their preparedness for the 
lecture (Q2), confidence levels (Q3), and preference toward using e-learning (Q4 and Q5). In other 
words, the module survey administered at the end of the e-learning module, which students com¬ 
pleted before the lecture, was to measure students’ predictions of their preparedness, confidence, 
and attitude toward the e-learning, and the post-lecture survey was to measure their evaluation of 
the effectiveness of the e-learning module on helping them prepare for the lecture and their attitude 
toward using it. The overall implementation procedure is illustrated in Figure 7: 


B. Results 

1) Overall results: Among 50 participants, not everyone completed all four activities (1. mod¬ 
ule instruction, 2. module test, 3. module survey, and 4. post-lecture survey). Forty-two of them 



Male 

Female 

Total 

Younger (<24) 

22 

4 

26 

Older (>24) 

19 

5 

24 

Total 

41 

9 

50 


Table 3: Descriptive statistics on participants’ age and gender. 


Pre-instructional e-learning module 

Module 

instruction 

Module test 
(cognitive 
preparedness) 

Module 

survey 

(affective 

preparedness) 


Classroom 

-► 

Post-lecture 

lecture 


survey 



(affective 



preparedness) 


Figure 7: The implementation of e-learning and evaluation in ENCR 245. 
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completed the module test (i.e., eight reviewed the module instruction but skipped the module 
test). The scores ranged from 66.67 to 100, and the mean score was 82.78 ( SD = 8.49), indicating a 
reasonable level of cognitive preparedness before classroom learning. Thirty-six of them submitted 
the module survey at the end of the module, and 42 of them submitted the post-lecture survey. 
Twenty-eight students submitted both the module survey and the post-lecture survey. Wilcoxon 
signed-ranks tests were used to check reliability between the two complete sets of nonparamet- 
ric survey data [40], The results of the module survey and the post-lecture survey, as well as the 
Wilcoxon test results, are summarized in Table 4. 

Both surveys revealed that students thought that the e-learning module had helped them pre¬ 
pare for their classroom learning. Students felt very prepared for listening to the classroom lecture 
after having finished the e-learning module (M = 4.06, SD = .89 from the module survey, and M = 
4.05, SD = .76 from the post-lecture survey). Their confidence levels indicated in the two surveys 
were high as well (M = 3.94, SD = .89, and M = 3.88, SD = .70, respectively). They also expressed 
positive reactions toward studying with the e-learning module (M = 3.31, SD = 1.14, and M = 3.48, 
SD = 1.10, respectively). Wilcoxon tests confirmed that the two sets of data on the first 3 items 
were not significantly different. However, the mean scores of the question asking if they would like 
to use more e-learning showed a significant increase from the module survey (N = 28, M = 3.29) 
to the post-lecture survey (A/ = 28, M = 3.68), Z = -3.05, p < .01. This result might be because 
after the lecture, they saw the value of the e-learning module and became more interested in using 
it. Students thought that the length of the program was about right or slightly longer than they 
would have liked ( M = 3.31, SD - .58 from the module survey). The post-lecture survey included a 
question asking when they would prefer to use an e-learning module. Sixty-one percent of them 
said that they would prefer to use it before the lecture rather than after the lecture, 19.5% of them 
would prefer to use it after the lecture, and 19.5% of them would prefer to use it both before and 
after the lecture. 


Question 

Module survey 

(N = 36) 

Post-lecture survey 

(IV =42) 

Wilcoxon test 

(N = 28) 

Mean 

SD 

Mean 

SD 

Feel prepared 

4.06 (4.00) 

.89 (.86) 

4.05 (4.11) 

.76 (.74) 

z = — .78, p > .01 

Feel confident 

3.94 (3.93) 

.89 (.72) 

3.88 (3.96) 

.70 (.69) 

z = -.24, p > .01 

Like e-learning 

3.31 (3.29) 

1.14(1.15) 

3.48 (3.57) 

1.10(1.14) 

z = —1.89, p > .01 

Like to use more e-learning 

3.31 (3.29) 

1.06(1.05) 

3.60 (3.68) 

1.12(1.25) 

z = -3.05,p < .01 


Note. The Mean and SD values shown in parentheses are for the 28 complete sets of data. 

Table 4: The results of module survey, post-lecture survey, and Wilcoxon test. 
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2) Age difference: A t-test was used to compare the difference in module test scores of two different 
age groups. The difference between the two scores was significant, t (40) = -2.22, p < .05; older students 
(M = 85.55, SD = 9.68) performed significantly better on the module test than younger students (M 
= 80.00, SD = 6.14). Mann-Whitney (M-W) U tests were performed to compare age differences on the 
nonparametric data obtained from the surveys [40], The differences in their affective preparedness (as 
a function of age) found in the module survey and the post-lecture survey were not significant; however, 
a pattern seems to exist that younger students reacted more positively to studying with the e-learning 
module than older students (see Table 5). This difference may reflect age differences in preference or 
familiarity toward online environments in general. It may also imply a potential benefit of using an 
e-learning strategy particularly for younger students who, based on the module test scores, seem to 
need more learning support to improve their academic performance to the level of older students. 


V. EVALUATION II: REUSABLE E-LEARNING (SPRING 2007) 


A. Methods 

1) Purpose: In the spring semester of 2007, the same e-learning module was implemented as a pre-in- 
structional strategy in MSE 312 Mechanical Behavior of Materials, which is a junior-level course which includes 
the topic of the e-learning module, ‘Mechanical Properties of Metals.’ ENGR245 Introduction to Materials 
Science and Engineering is a prerequisite to MSE 312; therefore, all students who enrolled in MSE 312 are 
expected to have prerequisite knowledge about the topic covered in the e-learning module. However, on 


Question 


Module survey 

(N = 36) 

Post-lecture survey 

(IV = 42) 

Younger 

(IV =17) 

Older 

(N = 19) 

M-W test 

Younger 

(N =24) 

Older 

(N= 18) 

M-W test 

Feel prepared 

M = 4.00 

M = 4.11 

z = -.29 

M = 4.17 

M = 3.89 

z = -1.28 


SD = .93 

SD = .87 

p > .05 

SD = .76 

SD =.75 

p > .05 

Feel confident 

M = 3.88 

M = 4.00 

z = -.76 

M = 4.00 

M = 3.72 

z = -1.19 


SD = .85 

SD = .94 

p > .05 

SD = .59 

SD = .82 

p > .05 

Like e-learning 

M = 3.47 

M = 3.16 

z = -.69 

M = 3.63 

M = 3.28 

z = -.82 


SD = .87 

SD = 1.34 

p > .05 

SD = .92 

579 = 1.32 

p > .05 

Like to use more 

M = 3.47 

M = 3.16 

z = -.76 

M = 3.71 

M = 3.44 

z = -.50 

e-leaming 

SD = .80 

SD = 1.25 

p > .05 

SD = .99 

SD= 1.29 

p > .05 


Table 5: Age differences in the results of module survey and post-lecture survey. 
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a pre-test administered in the first week of the MSE 312 class that covered the prerequisite knowledge of 
mechanical properties from ENGR 245, students scored an average of 66.82% mastery level (SD = 26.14), 
which called for a need to provide remedial instruction. Therefore, the e-learning module in MSE 312 was 
used to evaluate the effectiveness of reusing the e-learning module in this advanced course in order to 
stimulate students’ recall of prerequisite knowledge and to prepare them for classroom learning. 

2) Participants: Nineteen students enrolled in the class, but one student did not complete the 
course. Three students were female. Due to the small sample size, age differences in the outcomes 
were not investigated. 

3) Instruments and Procedure: After the first use of the e-learning module in the fall of 2006, the 
e-learning module was modified to conform to the e-learning technical standard, Sharable Content 
Object Reference Model (SCORM) 1.2 [41] in order to run it on the Blackboard Learning Management 
System (LMS). Blackboard allows the tracking of students’ learning behaviors, including the number 
of attempts, the completion status, time spent, and test scores. Students were asked to complete the 
e-learning module posted on their Blackboard course web site prior to the classroom lecture that 
required knowledge of the topic on Mechanical Properties of Metals. The overall implementation pro¬ 
cedure is illustrated in Figure 8. 

B. Results 

One student started the e-learning module but left it incomplete; therefore, 17 sets of data were 
used for analysis. Students spent an average of 49.71 minutes on the e-learning module (SD = 45.22); 
14 students used it once, and the remaining three students used it two, three, or five times. Everyone 
performed with 80% accuracy or above on the module test, and the average module test score was 
92.06 (SD = 5.41), indicating a near-mastery level of cognitive preparedness on the specific topic 
before engaging in the advanced sequence of learning in the classroom. 

Students’ affective preparedness after completion of the e-learning module was also supported by the 
module survey data. Thirteen students submitted the module survey. Overall, students felt prepared (M 
= 4.15, SD = .80) and fairly confident (M = 3.62, SD = .96) to engage in a classroom lecture on advanced 
topics after having completed the e-learning module (see Figure 9 and Figure 10). However, once again, 


Pre-test 


Pre-instructional e-learning module 


Classroom 


Module 

instruction 

Module test 
(cognitive 
preparedness) 

Module 

survey 

(affective 

preparedness) 


lecture 


Figure 8: The implementation of e-learning and evaluation in MSE 312. 


SPRING 2010 


15 






















ADVANCES IN ENGINEERING EDUCATION 

Improving Engineering Students’ Cognitive and Affective 
Preparedness with a Pre-lnstructional E-Learning Strategy 


students differed in their attitudes toward the use of e-learning. While the majority of students liked the 
use of e-learning and wanted to use it more, three students did not like it at all (see Figure 11 and Figure 12). 
Also, interestingly enough, a correlation analysis revealed that their feelings of being prepared and confident 



1 (Not 2 3 4 5 (Very 

prepared well 

at all) prepared) 


Q1 Feel prepared 

Figure 9: A bar graph of Qi results. 
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Q2 Help me feel confident 


Figure 10: A bar graph of Q2 results. 
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Figure 11: A bar graph of Q3 results. 
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were positively correlated to the degrees of their preferences in using e-learning (see Table 6). Similar to 
the reaction obtained from ENGR 245 students, they perceived the e-learning module to be about the right 
length or a little bit longer than they would have liked (M = 3.46, SD = .66). 


VI. CONCLUSIONS 


A. Discussions of the Findings 

The main purpose of this project was to design and implement a pre-instructional e-learning 
strategy in an engineering curriculum to improve students’ cognitive and affective preparedness for 
classroom learning. The pre-instructional strategy was used to prepare students for classroom learn¬ 
ing, and the e-learning method was used to provide them with a self-paced learning environment. The 
effects of using the pre-instructional e-learning strategy were investigated in two different contexts: 
1) to prepare students in an introductory class to learn new material, and 2) to prepare students in 
an advanced class to recall prerequisite knowledge they learned in their introductory class. 



1 (Not at 2 3 4 5 

all) (Definitely) 

Q4 Like to use more e-learning 
Figure 12: A bar graph of Q4 results. 


Question Items 

1 

2 

3 

4 

1. Feel prepared 

- 

- 

- 

- 

2. Feel confident 

.77** 

- 

- 

- 

3. Like e-learning 

.60* 


- 

- 

4. Like to use more e-leaming 

.57* 

.80** 

.88 

- 


Table 6: Correlations among the data obtained from the module survey. 
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The overall effectiveness of using the pre-instructional e-learning strategy before learning new 
material is well summarized by the following student comment obtained from the module survey, 
“Overall, pretty effective in giving a heads-up about the lecture material.” However, results from the 
first evaluation study indicated that the effectiveness of, and attitudes toward, using a pre-instruc- 
tional e-learning strategy might differ depending on students’ age. After completing the e-learning 
module, older students were cognitively better prepared for classroom learning than younger stu¬ 
dents were, but younger students seemed to like the use of e-learning slightly (but not significantly) 
more than older students did. Such knowledge about learners’ characteristics enables instructors to 
tailor their instruction and to utilize more effective instructional strategies for different learners. It 
is particularly important to do so during the early years of engineering programs when the overall 
student success rates and retention rates are low. 

The use of the pre-instructional e-learning strategy in the advanced course was proven to be 
effective as well. Although students’ preferences in using e-learning differed, the second evalua¬ 
tion study revealed that students’ recall of prerequisite knowledge was close to the mastery level 
after they had completed the e-learning module. This result reinforces the premise of the project 
for implementing a RLO-based pre-instructional e-learning strategy in curriculum development to 
improve its cost effectiveness. In retrospect, had the pre-instructional e-learning strategy not been 
implemented before the lecture in the advanced class, the instructor would have had to spend time 
in her classroom, reteaching students to help them recall prerequisite knowledge. Or, students 
with low levels of readiness might have had difficulty following the information presented in the 
advanced class. 

B. Limitations of the Project 

This project has several limitations. First, the use of convenience samples limits generality of the 
evaluation findings only to Boise State University (i.e., a small population), and due to the small sample 
sizes, any significant or non-significant results of the inferential statistics obtained from the evaluation 
studies should be interpreted with caution. Second, the first evaluation study revealed age difference 
in the module test scores; however, the result could have been influenced by other confounding factors 
such as other learner characteristics including gender, students’ GPA, access to computers, and the 
number of credits enrolled, to name a few. For example, gender is another learner characteristic that 
deserves attention. However, only nine female students participated in the first study, which was not a 
sufficient number of subjects for evaluating gender differences with inferential statistics. Third, it was 
not feasible to use a control group in the evaluation studies to make a between-group comparison in 
addition to the within-group comparison. Whether the improved cognitive and affective preparedness 
outcomes reported in the evaluations were indeed due to the pre-instructional e-learning strategy or 
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influenced by other factors would not be certain without comparing the results to the control group’s 
outcomes. With this project being exploratory, focusing on the development and implementation of 
the project, a more rigorous experimental research was not designed into the evaluations. Fourth, the 
project team also experienced a learning curve during the project in terms of the technical aspects of 
designing and developing the e-learning module. For example, in retrospect, more practice examples 
should have been included in the e-learning module (Gagne’s events #6 Elicit Performance and #7 
Provide Feedback), and more cognitive feedback could have been employed in the module test. Dif¬ 
ferent designs of the e-learning module could produce different results. 

C. Lessons Learned 

From this project, the authors not only learned about the overall effectiveness of incorporating a 
pre-instructional e-learning strategy in an engineering curriculum on improving students’ cognitive 
and affective preparedness for classroom learning, but also increased their understanding of useful 
e-learning design strategies and fundamental principles of instructional design for future projects. 
Here is a summary of the lessons-learned. 

1) Fully incorporate an e-learning strategy in the course: Despite the general consensus, as 
discussed in this paper, that the pre-instructional e-learning strategy helped students feel better 
prepared and improve confidence levels, anecdotal data indicated that some students felt that the 
instructors were adding more work to the existing course workload and that they did not view it fa¬ 
vorably. Therefore, it would be better to incorporate the e-learning strategy into the coursework from 
the beginning of the course, by using them as scheduled assignments, while monitoring the overall 
course workload and maintaining the quality and quantity of subject material being covered. 

2) Do not replace the good ol’ book-reading assignment and personal coaching with e-learn- 
ing: E-learning is not a panacea. It is a strategy, and different learners use, or prefer using, different 
strategies to maximize their learning. Therefore, a pre-instructional e-learning strategy should not 
be used to replace other options such as reading a textbook or asking questions to the instructor 
or peers. The personal preferences are evidenced by the following comments that some of the par¬ 
ticipants of the studies made to the open-ended question in the module survey: 

“I prefer the classroom and the interaction with a real person.” 

‘‘Pretty cool, but nothing really beats a good book over the material.” 

“I would much rather read the book before class, but that may just be me.” 

With that in mind, e-learning that is redundant to reading a textbook should be avoided. Instead, 
e-learning should be designed with unique instructional methods, such as providing more interactive 
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learning stimuli, more application and analysis types of practice questions, and immediate feedback. 
E-learning has these capabilities, as explained in the following items. 

3) Design e-learning to maximize the benefits of self-paced learning and multimedia capabilities: 

Learner preferences also differ in the use of multimedia and the degree of learner control during the 
participation in e-learning. Several students’ comments in the module survey show their preferences 
in using multimedia elements and having control over the learning sequence: 

“I liked the visual aids. They were helpful in visualizing the content." 

“I liked it when arrow pointed to what was being talked about.” 

‘‘I like this. It helped because I could read this as many times as I needed and study at my 

own pace.” 

“I would include a button to either skip quiz questions, or go back and review previous quiz 

questions.” 

E-learning has such capabilities, and e-learning designers should incorporate them into their 
products to maximize the benefits of self-paced learning and to support learner preferences. 

4) improve ‘transfer-of-iearning’ with ‘real-world’ examples: The e-learning module used in this 
project was designed to facilitate the acquisition or recall of knowledge and comprehension levels of 
learning (knowing what). However, e-learning modules can be designed for different types and levels 
of learning. Since many students leave engineering programs due to a misunderstanding between 
the relevance of their early coursework and a future engineering career, e-learning modules could be 
implemented in a manner to aid in making the link between the early, general courses and a future 
engineering career. One method for forming such a link is to incorporate “real-world” applications into 
the e-learning design. To solve these “real-world” examples, students would have to utilize the concepts 
and principles being covered by a particular module. In other words, e-learning could be designed to 
provide not only declarative knowledge (knowing what), but also procedural knowledge (knowing how) 
and situated knowledge (knowing when/why) [42], Different instructional methods should be used 
for different types of learning contents, and the degree of "real-world” applications would increase as 
the amount of 'procedural’ and ‘situated’ knowledge increases (see Table 7). By instituting “real-world” 
examples in e-learning, a direct link could be made between the coursework and the problems being 
addressed by practicing engineers. In addition, these examples could improve students’ understanding 
of engineering careers and instill in them an underlying reinforcement of life-long learning goals. 

5) Increase the degree of reusability of LOs through a spiral curriculum: RLO-based e-learn- 
ing products are better designed and implemented within a spiral curriculum, which is a curricu¬ 
lum development method that Jerome Bruner, an educational psychologist at Harvard University, 
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Learning 

Contents 

Declarative knowledge 
(Knowing what) 

Procedural knowledge 
(Knowing how) 

Situated knowledge 
(Knowing when/why) 

Instructional 

Drill and practice 

Show and tell 

Scenario/problem-based 

Methods 


(demonstration) 

(simulation) 

Degree of 

Low 

Medium 

High 

“real-world” 




applications 





Table 7: Types of learning contents and instructional methods. 


proposed about a half century ago. Improving ‘readiness’ is the key in a spiral curriculum, as Bruner 
[43] explains in the following: 

Readiness ... consists of mastery of those simpler skills that permit one to reach higher skills. 
Readiness for Euclidian geometry can be gained by teaching intuitive geometry or by giving 
children an opportunity to build increasingly elaborate constructions with polygons. Or, to 
take the aim of the new, “second-generation” mathematics project, if you wish to teach the 
calculus in the eighth grade, then begin it in the first grade by teaching the kinds of ideas 
and skills necessary for its mastery later, (p. 29) 

A spiral curriculum would not only support the design and implementation of RLOs to be used in 
multiple courses but also facilitate the improvement of students’ cognitive and affective preparedness. 
It requires collaboration among the instructors of a series of courses to analyze reusability of contents 
and to design and develop RLOs. In their classrooms, students should be repeatedly informed that 
specific topics will be used as prerequisites in upper-level coursework, and they could use a variety 
of RLOs to improve learning. Doing so would also help students recognize “the cumulative power of 
learning” [43, p. 30], which in turn reinforces their continuous motivation to learn and their under¬ 
standing of the connection between the early subject matter and a future engineering career. 

6) The shorter the RLOs, the better: Because the goal is not just to develop reusable materials, 
but also to ensure that they will be effectively reused, it is important to consider how to motivate 
students to use the RLOs in both formal and informal learning contexts. In addition to including 
multimedia components and real-world examples as motivational strategies, the development team 
has agreed that the shorter the RLOs are, the more useful they become. A desirable length of each 
RLO is 30 minutes or less. That way, students can select and review the ones that they need in a 
short time, which will help sustain their motivation to use them again, as needed. 
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D. The Follow-Up 

Since the initial development and evaluations reported in this paper, ten additional e-learning 
modules have been developed on the topics of mechanical properties, crystal defects, phase dia¬ 
grams, eutectic phase diagrams, the iron-carbon phase diagram, dislocations and slip systems, and 
energy band diagrams. All e-learning products are SCORM-conformant, so that students’ learning 
processes are tracked through a SCORM-conformant learning management system, Blackboard. The 
e-learning modules have been implemented in ENGR 245 class (60-70 students each semester) and 
MSE 312 class (17 students each year) as a pre-instructional strategy. The modules are also made 
freely available via the college website for any students who wish to use them to prepare for their 
Fundamentals for Engineering Exam for Licensure. 
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APPENDIX A. MODULE SURVEY 


Q1. After completing this self-paced program, how prepared do you feel to listen to the 
classroom lecture on this module? 

Not prepared at all 1 2 3 4 5 Very well prepared 

Q2. Will this self-paced program help you to increase your confidence in understanding the 
classroom lecture? 

Will not help at all 1 2 3 4 5 Will help very much 

Q3. How much do you like studying with self-paced programs like this? 

Don’t like it at all 1 2 3 4 5 Like it very much 

Q4. Would you like to have more self-paced programs like this? 

Not at all 1 2 3 4 5 Definitely 

Q5. How was the length of the self-paced program? 

Too short 12 3 4 5 Too long 

Q6. Please write any suggestions regarding this e-learning module: 
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APPENDIX B. POST-LECTURE SURVEY 


Q1. Did you complete the self-paced program before you came to the classroom lecture on this 
module? 

_Yes _No (If your answer is no, stop here and submit your survey) 

Q2. Because you completed the self-paced program, how prepared did you feel to listen to the 
classroom lecture on this module? 

Not prepared at all 1 2 3 4 5 Very well prepared 

Q3. Did the self-paced program help you to increase your confidence in understanding the 
classroom lecture? 

Did not help at all 1 2 3 4 5 Helped very much 

Q4. How much do you like studying with self-paced programs like the one that you completed? 

Don’t like it at all 1 2 3 4 5 Like it very much 

Q5. Would you like to receive more self-paced programs that support classroom lecture? 

Not at all 1 2 3 4 5 Definitely 

Q6. When would you like to use a self-paced program? 

_I would rather use it before I come to the classroom lecture. 

_I would rather use it after I attend the classroom lecture. 

_I would like to use it both before and after the classroom lecture. 

Q7. Please write any suggestions regarding the use of self-paced programs: 
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